1 research outputs found
Cache-conscious Splitting of MapReduce Tasks and its Application to Stencil Computations
Modern cluster systems are typically composed by nodes with multiple processing
units and memory hierarchies comprising multiple cache levels of various sizes. To leverage
the full potential of these architectures it is necessary to explore concepts such as
parallel programming and the layout of data onto the memory hierarchy. However, the
inherent complexity of these concepts and the heterogeneity of the target architectures
raises several challenges at application development and performance portability levels,
respectively. In what concerns parallel programming, several model and frameworks
are available, of which MapReduce [16] is one of the most popular. It was developed
at Google [16] for the parallel and distributed processing of large amounts of data in
large clusters of commodity machines. Although being very powerful tools, the reference
MapReduce frameworks, such as Hadoop and Spark, do not leverage the characteristics
of the underlying memory hierarchy. This shortcoming is particularly noticeable in
computations that benefit from temporal locality, such as stencil computations.
In this context, the goal of this thesis is to improve the performance of MapReduce
computations that benefit from temporal locality. To that end we optimize the mapping
of MapReduce computations in a machine’s cache memory hierarchy by applying cacheaware
tiling techniques. We prototyped our solution on top of the framework Hadoop
MapReduce, incorporating a cache-awareness in the splitting stage.
To validate our solution and assess its benefits, we developed an API for expressing
stencil computations on top the developed framework. The experimental results show
that, for a typical stencil computation, our solution delivers an average speed-up of 1.77
while reaching a peek speed-up of 3.2. These findings allows us to conclude that cacheaware
decomposition of MapReduce computations considerably boosts the execution of
this class of MapReduce computations